Binary risk control - implementation batch 1#735
Merged
Conversation
Valentin-Laurent
force-pushed
the
binary-risk-control-v1
branch
from
July 30, 2025 09:03
f883ca9 to
13462fc
Compare
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR implements binary classification risk control in its simplest form, focusing on mono-risk and unidimensional lambda using thresholding on predict_proba. The implementation includes cleanup of existing risk control code and introduction of new binary classification risk control components.
Key changes:
- Remove unnecessary components from existing LTT procedure (lambda=None check, p_values output)
- Add support for array-based n_obs in p-value calculations for binary classification scenarios
- Implement BinaryClassificationRisk class with predefined instances for precision, recall, and accuracy
- Introduce BinaryClassificationController for threshold-based binary classification risk control
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| mapie/control_risk/ltt.py | Remove p_values return, lambda=None check, and add array support for n_obs |
| mapie/control_risk/p_values.py | Add array support for n_obs parameter in Hoeffding-Bentkus p-value computation |
| mapie/risk_control.py | Add BinaryClassificationRisk class and predefined risk instances |
| mapie/risk_control_draft.py | Implement BinaryClassificationController with threshold-based risk control |
| mapie/tests/test_control_risk.py | Update tests for LTT changes and add n_obs array testing |
| mapie/tests/test_risk_control.py | Add comprehensive tests for BinaryClassificationRisk instances |
| mapie/init.py | Remove risk_control_draft from exports |
Comments suppressed due to low confidence (2)
mapie/tests/test_risk_control.py:848
- The test should verify the specific condition that leads to None result (effective_sample_size == 0) rather than using a general elif clause. Consider adding an explicit assertion that effective_sample_func returns 0 when result is None.
elif result is None:
mapie/risk_control_draft.py:18
- The entire BinaryClassificationController class is marked with pragma: no cover, indicating missing test coverage. This class implements core functionality and should have comprehensive unit tests.
class BinaryClassificationController: # pragma: no cover
Valentin-Laurent
force-pushed
the
binary-risk-control-v1
branch
from
July 31, 2025 12:50
8f2a127 to
ed18964
Compare
Valentin-Laurent
force-pushed
the
binary-risk-control
branch
from
July 31, 2025 12:50
37c79dc to
4404ad1
Compare
Valentin-Laurent
force-pushed
the
binary-risk-control
branch
from
August 27, 2025 12:55
4404ad1 to
5811aa9
Compare
- Use BinaryClassificationRisk to compute risk - Use warning instead of error when risk is not controled. Throw error when predicting - Remove useless check on lambda=None in ltt_procedure - Remove useless p_values from ltt_procedure outputs - Add possibility to pass an array of n_obs to ltt_procedure and subsequent p-values calculations (needed for binary classification)
- Fix bentkus_p_value calculation - Fix and move higher_is_better logic in the same place - Implement unit test for BinaryClassificationRiskControl - Fix parametrizing of existing test
… positive predictions)
Valentin-Laurent
force-pushed
the
binary-risk-control-v1
branch
from
August 27, 2025 12:55
d08fbea to
e661adb
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## binary-risk-control #735 +/- ##
=======================================================
Coverage ? 100.00%
=======================================================
Files ? 56
Lines ? 6205
Branches ? 355
=======================================================
Hits ? 6205
Misses ? 0
Partials ? 0 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Valentin-Laurent
marked this pull request as ready for review
August 29, 2025 15:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.